Strong Minimax Lower Bounds for Learning

Strong minimax lower bounds for learning

Minimax lower bounds for concept learning state, for example, that for each sample size n and learning rule gn, there exists a distribution of the observation X and a concept C to be learnt such that the expected error of gn is at least a constant times V=n, where V is the vc dimension of the concept class. However, these bounds do not tell anything about the rate of decrease of the error for a...

Strong Minimax Lower Bounds for Learning

Known minimax lower bounds for learning state that for each sample size n, and learning rule g n , there exists a distribution of the observation X and a concept C to be learnt such that the expected error of g n is at least a constant times V =n, where V is the VC dimension of the concept class. However, these bounds do no tell anything about the rate of decrease of the error for a xed distrib...

Minimax lower bounds for estimation

Minimax lower bounds

Now that we have a good handle on the performance of ERM and its variants, it is time to ask whether we can do better. For example, consider binary classification: we observe n i.i.d. training samples from an unknown joint distribution P on X× {0,1}, where X is some feature space, and for a fixed class F of candidate classifiers f :X→ {0,1} we let f̂n be the ERM solution f̂n = argmin f ∈F 1 n n ∑...

Minimax Lower Bounds for Dictionary Learning from Tensor Data

This paper provides lower bounds on the sample complexity of estimating Kronecker-structured dictionaries for Kth-order tensor data. The results suggest the sample complexity of dictionary learning for tensor data can be significantly lower than that for unstructured data.